1995 Institute: Frontiers in Laboratory Practice Research

Analytical goals are the quantitative requirements that the product must meet. Goals must be specific and have a clear success/failure criterion. Ideally, goals should include a protocol and data analysis method. There are three categories of analytical goals. First, clinical acceptability is the total analytical error and total analytical error sources. Second, there are a variety of regulatory goals depending on the approval required (510k, PMA, PLA). Finally, there is a list of marketing goals spawned by the competitive nature of business. Setting goals involves: defining a metric, setting its target, and specifying a protocol and data analysis method. Whereas many of the metrics are defined, the target setting process is still difficult for manufacturers. Laboratorians may know what they need, but effective communication of these needs to manufacturers is lacking. To set targets, manufacturers perform surveys (open ended questions, multiple choice questions, focus groups, and conjoint analysis). They also use performance data for similar, released assays (CAP data, internal data, published evaluation data). Reviewing existing goals reveals inadequacies such as non existent goals, non quantitative goals, goals without a meaningful success/failure criterion, and unsupported goals. The goal setting process can be improved by deciding on and gaining experience with a metric, preparing cause and effect diagrams, and challenging existing goals. Goals and claims are different. Manufacturers have internal goals. Upon product release, these goals are transformed into “claims”, which may be different from the internal goals. Different manufacturers state claims differently, leading to confusion. Claims are: 1) The “typical data” claim Half of the customers are expected to observe better performance, and the other half, worse performance. 2) The “warranty” claim Here, all customers are guaranteed performance better than the limit. What is needed is a common vocabulary for the laboratorian and manufacturer. Introduction Manufacturers of diagnostic assays have a consideration. This paper focuses on goals key milestone in the product development before product release. They can be divided process: product release. The analytical into three conceptual categories: clinical performance goals they set are often quite acceptability, regulatory, and competitive different before and after product release. goals. Before release, goals are often called Lab results have error. Clinical (internal) specifications and, after release, acceptability goals define how bad the error product claims. Laboratorians never see the can be before it causes diagnostic problems. internal specifications. Sometimes there is Laboratory assays are regulated. For confusion as to which goals are under 266 1995 Institute: Frontiers in Laboratory Practice Research manufacturers, this means that assays must extract. Without a correct analysis and be FDA approved. Regulatory goals depend reporting procedure, interpretation of the on the approval required (510k, PMA, PLA). data will be difficult if not impossible. For a laboratory, regulatory agencies require Without protocols and analysis methods it acceptable performance on proficiency is unclear how to determine if a goal is being surveys. Hence, assays must achieve a met. For example, for a glucose assay with a certain performance level with proficiency range of 5-1000 mg/dL: do we need to survey controls. Finally, companies must be evaluate precision at 5 concentration levels, competitive to remain in business. This every 20 mg/dL, every mg/dL? Can we spawns a list of marketing analytical spike and dilute samples? If we dilute, what performance goals. should be the diluent? To test interferences, Definition of an Analytical Goal Analytical goals are a subset of the the goal, the point estimate? Its 95% quantitative requirements that an assay must confidence interval, its 99% confidence meet. The terms specifications, target interval, every data point within the goal? values, and requirements are synonyms of NCCLS evaluation protocols help address analytical goals. Goals must be specific and some but not all of these issues. Experience have a clear success/failure criterion (e.g., shows that agreement for these issues helps there must be a metric). Ideally, goals to prevent questions after the data have been should specify a protocol and data analysis collected. method. This assures that not only the right type and amount of information will be collected but also describes how the data will Constructing goals involves: be analyzed and reported. An analytical goal example is: the total precision CV should be defining a metric (e.g., % CV precision) less than 10% throughout the 50-500 mg/dL setting the metric’s target (e.g., < 5% range as determined by the NCCLS protocol CV precision) EP5. defining a protocol to evaluate the A protocol and analysis method is metric (e.g., 2 observations per day for recommended as part of a goal because the 20 days) analytical performance of an assay differs defining an analysis and reporting from directly measurable assay properties procedure for the metric (e.g., such as the size and weight of an instrument. ANOVA) Analytical performance cannot be exactly determined the true performance values Consider that most assays are developed (the “true state of nature”) can only be by manufacturers for sale to clinical estimated by experiments. Variation in the laboratories. Laboratories run the assays and experimental results prevents their direct provide clinicians with results. Clinicians use determination. The resulting data from these the results to help answer the question, experiments has information in it that a “Should I treat or not treat the patient?” properly designed analysis procedure will what glucose level(s) should be used? If bias is evaluated, what is the criterion for meeting